Probabilistic Characterization of Decision Trees Probabilistic Characterization of Decision Trees

In this paper we use the methodology introduced in Dhurandhar and Dobra (2006) for analyzing the error of classifiers and the model selection measures, to analyze decision tree algorithms. The methodology consists of obtaining parametric expressions for the moments of the Generalization error (GE) for the classification model of interest, followed by plotting these expressions for interpritability. The major challenge in applying the methodology to decision trees, the main theme of this work, is customizing the generic expressions for the moments of GE to this particular classification algorithm. The specific contributions we make in this paper are: (a) we completely characterize a subclass of decision trees namely, Random decision trees, (b) we discuss how the analysis extends to other decision tree algorithms, and (c) in order to extend the analysis to certain model selection measures, we generalize the relationships between the moments of GE and moments of the model selection measures given in Dhurandhar and Dobra (2006) to randomized classification algorithms. An extensive empirical comparison between the proposed method and Monte Carlo, depicts the advantages of the method in terms of running time and accuracy. It also showcases the use of the method as an exploratory tool to study learning algorithms.